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Abstract 

We study a learning rule based upon the temporal correlation (weighted 
by a learning kernel) between incoming spikes and the internal state of 
the postsynaptic neuron, building upon previous studies of spike timing 
dependent synaptic plasticity ([HOlini)- Our learning rule for the synaptic 
weight Wij is 



J-oo Jt-Ti „ 



r + s — tj^fi)u{T)dT T{s)ds 

where the tj^f^ are the arrival times of spikes from the presynaptic neu- 
ron j and the function u{t) describes the state of the postsynaptic neu- 
ron i. Thus, the spike-triggered average contained in the inner integral 
is weighted by a kernel r(s), the learning window, positive for negative, 
negative for positive values of the time diffence s between post- and presy- 
naptic activity. An antisymmetry assumption for the learning window 
enables us to derive analytical expressions for a general class of neuron 
models and to study the changes in input-output relationships following 
from synaptic weight changes. This is a genuinely non-linear effect f|16|'). 

1 Introduction 

This paper deals with the question of incorporating correlation based learning 
mechanisms in formal neuron models. According to the so-called Hebb rule, 
such correlations are encoded by synaptic weights, and learning is considered 
as a mechanism implementing this. More precisely, the decisive feature of the 
mechanism of synaptic plasticity is its response to temporal correlations in the 
input-output pattern of the postsynaptic neuron. The pattern of STDP, spike 
timing dependent synaptic plasticity, emerged both from profound theoretical 
investigations of Gerstner, Kempter, van Hemmen, Wagner, and others (1313 
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EEIIHIE]), inspired by discoveries about the processing of spatial information 
in the auditory system of the barn owl, from detailed experimental studies 
(see IHIIHIEI^J), and from carefully set up computer simulations Put 
simply, the result is that a synapse gets strengthened when a presynaptic input 
arrives shorthly before the generation of a postsynaptic spike, and that it gets 
weakened if the two events occur in reverse temporal order. This also solved one 
of the problems for the implementation of Hebb's rule in older network models 
that followed a continuous dynamics, namely that one needed an additional 
mechanism that was either ad hoc or non-local to prevent the synaptic strengths 
from growing without bounds (see \17\). 

The theoretical analysis, however, ultimately needed a neuron model that was 
linear in the sense that the neuron's activity, spiking probability, or spike rate 
depended in a linear manner on the input received. The resulting learning rule, 
while based on the correlation between in- and output, then did not in turn 
affect that correlation. The computer simulations of Song et aL^ni) however, 
demonstrated that such learning based on temporal correlations in a non-linear 
neuron model could sharpen those correlations and thus make the operation of 
the neuron more efficient in this sense. It is the purpose of the present article to 
provide a framework within which temporal correlation based learning and the 
resulting changes in those correlation patterns can be analyzed for general non- 
linear neuron models.^ For this, we shall need to make one crucial symmetry 
assumption about the shape of the learning window which, while not in direct 
qualitative contrast with the neurobiological findings, apparently is not strictly 
quantitatively valid. If we take as our state function the firing rate of the 
postsynaptic neuron (or some function of that firing rate), our model is a hybrid 
between a timing and a rate dependent model. Still, as argued in |18| . for 
capturing the full wealth of experimental findings, probably models are needed 
that include a more refined relationship between timings and rates. For those 
reasons, our results can only be considered as a somewhat crude approximation 
of the underlying neurobiological reality, but we hope that the elegance of the 
theoretical principle will still provide us with useful insights. It remains the task 
of deriving this principle from information theoretic considerations. 

2 Temporal correlations and the learning rule 

We consider the synaptic strength Wij from a presynaptic neuron j to a post- 
synaptic one i. i shall stay fixed over the entire course of our analysis, and so 
the dependence on i could well be omitted from our symbols, but we find it 
useful to include it nevertheless. By way of contrast, we shall need to compare 
the effects of several presynaptic neurons, and so the index j is indispensable. 
The learning rule then is a differential equation for Wij as a function of time t, 

^ While we can treat non-linear neuron models, however, the learning rule employed is itself 
linear in the sense that it assumes a linear dependence of the weight changes on some internal 
state function; that state function could be the firing frequency, but it could also be some 
non-linear function of it. 
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depending on the temporal correlation between pre- and postsynaptic activities, 
i.e., the ones of the neurons j and i. 

The aim that the learning rule should depend on temporal correlations between 
pre- and postsynaptic activity and the fact that the activity and the learning 
dynamics take place on different time scales makes it necessary to specify the 
relation between those time scales. Based upon [30], it is carefully explained 
in |B] that the time scale Ti on which the synaptic weight changes are analyzed 
should separate the time scale of the neural spikes and the typical interspike in- 
tervals from the one where the effects of learning become visible. More precisely, 
both an "ergodic" and an "adiabatic" hypothesis is employed. The first one al- 
lows to average over randomness, that is, over repeated trials, or, equivalently, 
over the time course of a single, sufficiently long, trial whereas the second one 
allows to average over time, that is, to work with quantities that can be assumed 
to be constant on a time interval of length Ti . Here, we shall also employ the 
latter, the adiabatic hypothesis, but use the former only in a somewhat weaker 
form. 

The presynaptic activity is given by the spike train 



produced by j. Here, the tj^^ are the firing times of neuron j in the time 
period under consideration, and we use the standard (5-function formalism. For 
the postsynaptic neuron i, instead of directly working with its spike train, we 
employ a state function u{t). Since this is fundamental for our approach, we 
should carefully discuss the underlying reasoning: 

• While a presynaptic spike is an event that cannot be further analyzed 
and has to be considered as externally caused, the activity and the firing 
pattern of the postsynaptic neuron depend on the inputs received from 
many other neurons, not only from the one forming the synapse under 
consideration. Therefore, it is appropriate to work with some average for 
the total input of the neuron and to consider its state as the dynamical 
response averaged over the total input. 

• In contrast to the presynaptic spike, a postsynaptic spike should not be 
considered as an independent event. We rather need to employ a model 
for the dynamic activity of the neuron in response to its input, i.e., we 
need to suppose some deterministic relationship between input and output 
- not necessarily between individual spikes, but for example between an 
incoming spike and a postsynaptic spiking probability. The effect of a 
single presynaptic spike may be very slight, but nevertheless it should 
make some definite contribution to the state of the postsynaptic neuron. 
That contribution may well depend on previous contributions from other 
presynaptic neurons, and on the present postsynaptic state itself, and this 
will then lead us to non-linear models. 

We shall discuss various possibilities for the state function u below. At the 
moment, we need not specify it any further. The next ingredient in our learning 
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rule is the shape T{s) of the time window that describes how the time difference 
s between pre- and postsynaptic activity influences the synaptic weight change. 
It is generafly agreed in the Hterature on STDP that both experimental evidence 
(see ^U] for a review) and conceptual reasoning ([2113101) lead to the following 
qualitative behavior: 

1. r(s) = whenever the absolute value of the time difference s between pre- 
and postsynaptic spike is large. 

2. r(s) > whenever the presynaptic spike shortly precedes the postsynaptic 
one, i.e., when s is negative and of small absolute value. ^ This effect 
has been called LTP (long term potentiation). This is interpreted as a 
manifestation of causality, in the sense that the presynaptic spike can 
then be considered as contributing to the postsynaptic one. 

3. r(s) < whenever the presynaptic spike occurs shortly after the postsy- 
naptic one, i.e., when s is positive and of small absolute value. This effect 
is called LTD (long term depression). 

In addition, we shall need one further assumption on r(s): 

4. r(s) is balanced in the sense that 



This assumption will be needed subsequently to convert an integral of a 
neuronal state function u against the time window function T into an in- 
tegral of a differences of states against F restricted to positive arguments, 
in other words for evaluating and weighting the effect that an input causes 
on the state of the postsynaptic neuron. In our analysis below, we shall 
mostly work with a state function that represents a firing probablity of 
the postsynaptic neuron. In that situation, the function F can be inter- 
preted as the neurophysiological learning window, and it then becomes 
an issue whether that balancing condition is experimentally supported. 
We shall summarize some of the findings in this direction shortly. Our 
framework below, however, also allows us to consider state functions that 
do not directly correspond to firing probabilities, but perhaps some other 
internal quantities, or that subject such a firing probability to some non- 
linear transformation. Under such more liberal conditions, the balancing 
assumption employed here is still compatible with a non-zero integral of 
the neurophysiological learning window. 

While, returning to spike probabilities, this balancing assumption docs not 
seem too far from the neurophysiological findings, it is not strictly sup- 
ported by them. At least, at present the available data seem to be mixed 
on this issue (see the discussion and the references in ^Hl)- /_oo ^('^)^'^ 

■^For an exception where there has been found an additional negative window for a certain 
range of negative s, see 1191 . 
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has not always been found to be strictly 0, but at least it seems to be 
rather small in most situations. Recent experimental results that report 
a non-zero integral are, for example, described in |12[ I13L ITU I18p . typ- 
ically with a longer window and a slight dominance for LTD, whereas 
those finding similar time frames for strenghtening and weakening report 
approximate equality between the two effects or a slight dominance for 
LTP 13 . In any case, since the detailed electrochemical processes at 
a synapse before and after reception of a spike are different, there cannot 
be any direct biophysical reason for 0. At most, there may be a very 
indirect reason for an approximate validity of Q . Namely, if that relation 
should turn out to be most advantageous for the processing of information 
in neuronal systems, it might be implemented by the forces of evolution. 
This, however, may seem a little far-fetched in the present context, and 
so, this assumption can be defended here only on the basis of its analytical 
utility. The condition (O, r(s) ~ 0, can, however, be also consid- 
ered as a normalization that prevents the synaptic weights from growing 
without bounds when subjected to Hebb type learning rules. As argued 
in however, for that purpose a (small) negative value of that in- 

tegral would be better, in order to weaken inputs that do not consistently 
contribute to postsynaptic firing, but only by chance occur at about the 
time of that firing. Uncorrelated pre- and postsynaptic activities would 
then induce a weakening of the synapse. By way of contrast, under our 
assumption, if we take as our state variable for the postsynaptic neuron 
its firing rate, statistically independent pre- and postsynaptic spike trains 
produce no weight change. However, since we may use a non-linear func- 
tion of the firing rate, we still possess a certain flexibility here. The really 
serious issue is that our symmetry assumption prevents us from making 
a rate-dependent distinction between LTP and LTD. Such a distinction, 
i.e., that at low rates, LTD dominates, whereas at higher rates, LTP does, 
has been discussed in JS]. Also, these authors have found that a model 
that includes rate, timing, and cooperativity captures some experimental 
findings better than one that solely depends on the relative timing of pre- 
and postsynaptic spikes. In their model, only nearest spike interactions 
count, and LTP wins over LTD. In partial contrast to this, Froemke and 
Dan|14| found that synaptic modification depends not only on the relative 
timing of pre- and postsynaptic spikes, but also on the spiking patterns 
within each neuron. In particular, the first spike within a burst is found 
to be the dominant one. 

As a consequence of our symmetry assumption (jJJ, there exists a trans- 
formation s*{s) of the positive reals onto themselves such that for s > 0, 

■^It is not clear to the present author, however, how accurate the estimates for the integral 
are. For example, in 1101 . a negative integral is computed on the basis of fitting the data by 
exponential functions, and the negative integral then results from a longer decay time of LTD. 
That the data are described well by an exponential function, however, is not entirely obvious 
to the present author, and fitting them with a different type of function might well lead to a 
different value and sign for the integral. 
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r(— s*(s))^ = — r(s), and consequently, for functions v (assuming nat- 
ural integrability assumptions), 



■0 



/ 



■ oo 



v{a)T{a) da 



v[~s*[s)){~T{s)) ds. 



(3) 



(The formally simplest case is the one where T is an odd symmetric func- 
tion, in the sense that r(— s) = — r(s) for all s; in that case, of course 
s*(s) — s.) The point here is that in general the time courses realizing 
LTD and LTP are different (see [2111 for a recent survey of the molecular 
mechanisms underlying these processes). By the symmetry assumption on 
r the total, time-integrated effects of the two processes are of the same 
magnitude, but stretched differently over time. The function s*{s) rescales 
the latter process to make it symmetric to the former. 

The idea for our learning rule is now quite simple: We convolve an input- 
output correlation spike-triggered average with the learning window function T 
to obtain a differential equation for the synaptic weight Wij : 



where the sum extends over all spikes between t and t — Ti (and where we neglect 
the discontinuities as a function of t when a spike happens to fall precisely on 
one of these boundary points, because Ti is supposed to be much larger than 
the typical interspike interval). The spike-triggered average occurring here, that 
is, the inner integral, is known in the literature as stimulus-response correlation 
or peri-stimulus time histogram. In other words, we stipulate that the synaptic 
weight change is proportional to an input-output correlation weighted by the 
kernel T . The proportionality factor e is supposed to be small, to make learning 
into a slow process that shows significant effects only on a time scale larger than 



Following we may use the adiabatic hypothesis to extend the integration 
boundaries for s to ±oo. Namely, when tj^^ = t — Ti+xTi for < a; < 1, s in the 
integral varies between (x — 1)T/ and xTi, and for any given x and sufficiently 
large T;, the integration bounds can be taken as ±oo because r(s) = for 
\s\ » 1. Thus, we may write 




-oo J/ Jt-Ti ^ 



Ti, as in OEIEI. ^ is 





w.. 




W r E "fe^A' - s) T{s)ds 

J~oo ^ 
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s)){-ns))ds 
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because of the symmetry assumption 4 above, and this receives a positive con- 
tribution if u is larger after the incoming spike tj^^ than before 

= ^tE/ / uit,,^ + r)dT{^Tis))ds (7) 

J/ „ Jo J-s 

which receives a positive contribution if u is increasing between fj.^ — s and 



3 Neuron models 

In order to gain some prehminary understanding of the learning rule we 
briefly consider the case where the state function u is also given by a spike sum, 
u{t) — J2u ^i'^ ~ ^j,!')- Inserting this into 10} yields 

— / (5(t + s - tj^fj)5{T - ti^^)dT T{s)ds 

/■°° 1 

= e^Erfe,M-^-.)- (8) 

Thus, the learning rule changes the synaptic weight by a weighted sum of spike 
time differences, as it should. 

In order to gain deeper insights, however, we need to assume a systematic re- 
lationship between input and output, and the approach taken here is that the 
state of the output neuron i is described by a state function u whose value 
then is determined by the inputs received. In the preceding example, we could 
insert a functional relationship that determines the difference in spike times 
— U,v as a function of the synaptic input received, and thus, assuming the 
presynaptic activity as given, of the synaptic weight Wij. This will then lead 
to a differential equation for the weight change in terms of that - and perhaps 
other - synaptic weights. Depending on that functional relationship, the weights 
in the model may then become unbounded or, which would be more pleasing, 
converge to some stationary value. The mechanism for the latter is that an 
increased weight might decrease the spike difference to so that then there will 
be no further increase since r(0) — 0. The analysis is rather straightforward and 
therefore omitted. We point out, however, that a vanishing, or even negative, 
integral of the learning kernel T does not automatically imply that the synaptic 
weights stay bounded in such models. Namely, once a certain weight Wij has 
grown so large that any presynaptic input triggers a postsynaptic spike, then, 
unless that spike occurs without any time delay simultaneously with the input, 
the weight will keep growing, unless some input from a different synapse causes 
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spikes shortly before the present input. Conversely, however, once our weight 
is so large, it will then exert that negative effect on other synapses rather than 
being dampened by those. 

We now turn to a more systematic analysis of how assumptions about the func- 
tional relationship between the input and the state u of our postsynaptic neuron 
drive the weight changes in our setting. In fact, we can insert any neuron model 
in (O that is given either by an explicit expression for the state function u or in 
the form of a differential equation for that state function. We start with the first 
possibility and consider a model where the state of our neuron i is computed as 

k v 

Here, the t\^^v are the spike times of the incoming neurons fc, and k is a synaptic 
transfer function. For causality, k needs to satisfy the requirement that K(r) — 
for r < 0, i.e., an incoming spike can only influence the state of neuron i after 
it occurred. Reasonable choices for k that have been discussed in the literature 
are (see pp for a general treatment): 

1) A function that sets in sharply upon the arrival of a spike at time and 
then decays exponentially: 

Ko(< - := eM~^-^^)H{t - tj.^), (10) 

T 

with the usual Heavyside function H. The derivative of this function is given 
by 

ko{t ~ tj,,,) = S{t ~ i,- ,J - - exp(-^^)i/(i - tj.^). (11) 

T T 

2) The so-called a-function that starts linearly upon arrival of a spike and again 
decays exponentially: 

'^lit - t,,^) := eM-^-^^)H(t - t,.^). (12) 

T T 

3) Both kq and ki have the disadavantage that they do not return to in finite 
time. This means that an incoming spike will have some effect forever, even 
though this effect of course decays exponentially. For that reason, one might 
prefer a decay to in finite time, for example a linear one: 

for < < 1 (13) 

T T 

and otherwise. 

The function / translates the synaptically weighted input sum into some acti- 
vation of neuron i; it could stand for its firing rate, probability, or propensity. 
In those cases, one would assume that it is monotonically increasing, i.e., has a 
positive derivative. 
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With this neuron model, © then becomes 
1 

- fiYl H - tk: + s*{s)))) T{s)ds. 

k v 

Since V(s) < for s > 0, the contribution to this expression at s > is positive 

if /(Efc w'^fe Ej. Kfe.M - tkM + s*)) > fCLk'^^kJ2^'^{tj,tJ^ - ^k,,^ - s)), i.e., if 
the state of i is larger after the spike at tj^^ than before. We recall that k{(j) 
vanishes for cr < 0, and that (for the choices kq or K2^ say) it is a decreasing 
function of cr > 0. Thus, if i^^^ — tk^y + s* > Q > tj^^ ~ tk^y — s, i.e., if 
— s < tk.i, < tj^^ + s* , we can expect a positive contribution, whereas if the 
spike of k comes too early, tk^i, < tj^ — s, we obtain a negative contribution. In 
other words, for a spike of neuron j to strengthen the synapse to neuron z, it 
needs to be good at anticipating or "predicting" other incoming spikes that in 
turn increase the state of i and thus its likelihood to fire. We thus confirm the 
conclusion of the previous investigations [2 El EIP ■ 

We now use one of the transfer functions kq or K2 for k; the modifications 
required for the choice ki will be obvious. We obtain from {Tj) 

" ^ / ^ y!/'(y!wjfc VK(tj-^ -ifc,^, -r)) (15) 

Wii ^{-S{tj^f_, - ti^p - r) - kitj^^ - ti^p - T))dT T{s)ds 
I p 

(where in contrast to the derivative k stands only for the smooth part) 

which in turn equals 

yi^^iyiwcy,w,ky^n{ti,p-tk,u)) (le) 

M i p k 

+ / /(y^ Wik K(tj,^ - tk,v - T))k{tj^^ - U^p - T)dT) {-T{s))ds. 

k V 

Note that we have shifted the minus sign to the F-tcrm. The first term here 
counts the number of spikes of ] with appropriate weights. In the second one, 
since k is decreasing, k yields a negative factor. 

We now turn to neuron models where the derivative of the state function u 
is given in terms of the input and some inherent dynamic features. An example 
is the leaky integrate-and-fire neuron that is described by 

= -ciu(i) + X! '^^'^ X! ~ ^'=>'^) ~ X! ~ ^^"^^ 

k V ti,A 
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for some constant ci that determines the time scale of the decay.''' Uth is the 
value of the spiking threshold, and the ti^x are the spiking times of our neuron 
i. We may then insert H17|l into {TJ to obtain the learning rule for the leaky 
integrate-and-fire neuron. The analysis can then proceed as for the above state 
function / where we encountered the derivative of the transfer function kq or 
K2, except that the signs get reversed because an output spike gives a negative 
instead of a positive contribution. The model is not entirely satisfactory because 
the output spike time is not explicitly determined as a function of the input. 
Also, the discontinuous resetting of u to upon the emission of a spike cannot 
directly be approximated by a smooth dynamics for the scalar variable u when 
u is given as a solution of an ODE, since a solution of such an ODE can only 
exhibit monotonic behavior when continuous. We can overcome this problem, 
however, by introducing an internal state variable d that takes its values in the 
unit circle instead of the interval [0, uth] so that u = is a 2-1 function. An 
example of such a model is the iJ-neuron introduced and studied by Ermentrout 
and Gutkin |^ |221 (see also PP for a general discussion) . We can set things up 
in such a manner that = corresponds to the rest point u = and = tt 
to the firing point u = Uth of our neuron. The neuron model then consists in 
expressing the time derivative ii of u = u{'d{t)) as a function of i?(t) and the 
input / (= J2k J2u '^(^ ~ as always), 

u=^^^Hm,m)- (18) 

We can then describe the generation and emission of a spike in a continuous 
manner. What is essentially needed is that the derivative ^ is positive in the 
upswing phase < d < it and negative in the downswing phase tt < < 27r 
(where, of course, 27r is periodically identified with 0). The precise shape of $ 
will, of course, depend on the biophysical model employed, but this is not our 
present concern. Qualitatively, the generation and emission of a spike and the 
subsequent resetting of a neuron then is described by a hump function v{t — ti^\) 
that can be made narrow, for example v{t) > precisely for |r| < 6q for some 
small So > 0, with i;(0) = uth- We then have the conservation J_°g^ v{T)dT = 0. 
Thus, a spike that occurs before the input at time tj,^ then leads to a negative 
contribution in Q for all s with U^x — tj^^ — (5o < < U,\ — tj.ii + and to a 
vanishing one for other values of s.^ Conversely, if tj^^ < U^x, we get a positive 
contribution for ti^x — tj^fj, — So < s < U^x — tj^p, + So and a vanishing one for 
other s. 

^Perhaps this example is not so good for the present setting, however, because in the 
integrate-and-fire neuron model, u denotes a membrane voltage and cannot be interpreted as 
a firing probability. 

^Let us point out that s is an integration variable, and so, this does not require that input 
and output spike appear virtually at the same time to have an effect. Rather, the function F 
is only evaluated at a narrow range of its arguments s, the extreme case of course being the 
one described in ISl . 
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For the subsequent analysis, a model of the qualitative form 



it = F{§) + G(i9) <^ - 



(19) 



fc V 



is particularly useful. Here, F can stand for a leakage term that then is always 
non-positive, whereas G{d) should be positive before the spike (i.e., for < ■(? < 
tt) and negative afterwards so that u can return to its rest value. A spike is 
then built up when the r.h.s. of H19() is positive for a sufficiently long time. For 
that model, {Tj) becomes 



The first sum here again extends over all spikes that come in from neuron j with 
t — Ti < tj,^ < t. It may then happen that the argument tj,^ + r in the integral 
is larger than t. This violates causality. However, as argued in Section 4 of 
0, those arguments are negligible because the learning window can be assumed 
much smaller than Ti. 

Let us assume for the moment for the sake of the discussion that the state 
variable -d, being the result of many incoming spikes, varies so slowly that we 
may replace d{t + r) by ^{t) for — s < r < s* . Then, the contribution of the 
spike at tj^n is positive when it occurs for < ?? < tt, since the integrand is 
positive in that case, i.e., when the neuron i increases its state variable towards 
the firing point, and negative in the interval corresponding to returning to rest 
after firing. The first term again counts the incoming spikes of j, weighted with 
the value of F{-d). Turning to the second term, and using the transfer function 
kq or K2, by the properties of n again, that contribution for the spike of neuron 
k at tk,v is strongest in absolute value for — tk.i^ + s because it can then 
exert its effect during the whole interval [— s, s*]. Thus, as a consequence of the 
decay of k after the initial pulse, it is best for the spike of neuron j to occur 
shortly after the one of neuron k, provided this happens at a time when i is 
responsive to incoming spikes. If k = ki instead, then the optimal delay of the 
spike of j after the one of k is even longer as the effect of the spike of k exhibits 
itself most strongly only after some delay. Returning to kq or K2, the effect of 
the spike tk,i, is small, if it occurred too early since then its contribution during 
the interval [tj^^ — s, tj^^ + s*] is weak, while it is also small if it comes too late 
because then its influence starts too late. If k decays to very rapidly, then it 
does not make that much of a difference anymore at which point in the interval 
[tj,fi — s, tj_^ + s*] the spike of k occurs. In any case, because it is still somewhat 
better for a spike of k to occur slightly before than after the one of j, we can- 
not conclude in the present model that it is advantageous for j to anticipate or 
predict the spikes of other incoming neurons, but rather to exploit the effects of 



(t) 



(20) 



e 




+G{^{tj^^ + t)) Y ^(^J.M - ik,t. + T)dT) {-T{s))ds. 



k V 
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those spikes and to bring i even closer to firing. In any case, a spike of j leads 
to an increase of the weight Wij when it occurs at a time when the state of i 
increases from the resting to the firing value, and to a decrease otherwise. 
These conclusions do not change significantly if we no longer replace ■d{t + r) 
by 'd{t) for — s < r < s*. The term F{d{tj^f_i + r)) is again straightforward 
to analyze. In the second term, the quantity J^k '^ik ~ ^fc,!^ + ''') 

now multiplied by the varying term G(z?(tj.^ + r)) which we assume here pos- 
itive when tj_^ + T is smaller than the firing time ti, and negative afterwards. 
When then the incoming spike at tj^^ occurs before the firing time, the term 
'^i^j,^t ~ ik.u + t) then receives a positive factor as long as t^^u < + t < U, 
and a negative one afterwards. If for example tj,^ — U, then spikes of k at 
tk,u < lead to a positive effect as long as r < 0, and to a negative, albeit 
smaller one, afterwards. Spikes at tk^v > tj^fj, have a negative effect for all r, al- 
though over a shorter span of time. While the precise contributions will depend 
on the detailed shape of the learning window F, as well as on the actual spike 
train tk^v, the positive and negative effects should about balance each other for 
= til while if for example tj^^ > ti, this balance will be shifted towards the 
negative ones. 



4 Input-output relationships 

In the preceding, we have investigated how the temporal relationships (correla- 
tions weighted with a temporal kernel) between pre- and postsynaptic activity 
drive the learning process as incorporated in synaptic weight changes. We now 
address the question of how in turn these correlation patterns change as the 
result of learning. In order to study this, we should separate the effects of the 
learning dynamics from the ones of the activity dynamics. As learning leads to 
changes in the dynamic behavior, we should assume that we start from a situa- 
tion where without learning, i.e., when considering solely the activity dynamics 
and assume that the synaptic weights are constant, the neuron responds to sta- 
tionary input in a stationary manner. In other words, without learning, the 
activity dynamics should always reach some state where the neuron responds to 
the same input in the same manner. In particular, without learning, the rela- 
tionship between input and output should be constant in time. This assumption 
then allows us to study the effects of learning on this relationship. We proceed 
to do so. The relationship between input and output determining the synaptic 
weight change in our learning rule was computed or estimated in ((^Jl, ©by 



cut) : 
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f°° 1 /•s*(s) 

= Ey tJ^ u(i,-^ + r)dr(-r(s))ds, (21) 

and we now wish to compute how this expression changes in time in response 
to learning. We obtain 

cut) - cut -h) 

/ u{tj.^+T + cr)d<jdT {-r{s))ds. (22) 

Jo J-s J-/i 



E 



E 



Letting h tend to 0, we obtain 



cut) = J2 1^ Y, "^^^^'^ + ^^'^^ {-ns))ds. (23) 



1 



Without learning, by our stationarity hypothesis, this expression should vanish. 
Looking for example at the second to last line in (|22|l . without learning, the 
integral over the derivative of u should be invariant under a time shift of the 
argument, here by the amount h. This means that for the above integral over 
il, we only need to investigate how the integral over the derivative of u changes 
by the learning dynamics when increasing the time by h. We study this for the 
model ifT^ . 

u^F{d) + G{^)Y,w^kY.n{t-tk,u)- (24) 

k V 

In order to simplify our notation, we abbreviate 'Ylik ''^ik X^i/ '^(t ~ tk.y) as wk; 
thus w stands for the synaptic weight under consideration. When subjected to 
a variation w of w, 1)24(1 varies by 

■^ii w = (F'id)^ + G'{d)^wn + G{^)k)w (25) 
dw dw dw 

(here, ' denotes a derivative with respect to i).)^ For the sake of the discussion, 
we separate the indirect contribution coming from the variation of the state 
variable •& in response to the change of the weight parameter w from the direct 
effect of this weight change on il as given by the last term in H25|) . The latter is 

G{i9)Y,Wrk{t)J2'^(t-tk,u)- (26) 

k V 

Inserting this into H22|l , we then obtain for the effect of this term on the weighted 
input-output correlation 

E/ r/ (27) 



^ Since we shall study effects on the time scale T; which is independent of e, the linearization 
performed here constitutes a valid approximation for small enough e. 
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Wik {tj^^ + X! ~ ^f'-^" + '^)'^'^ (-r(s))ds. 

k V 

From (EOl, we obtain a coarse estimate for this temporal change then as the 
average sign of 

(F(^W) + G(^(<))/(i))G(^(i)) (28) 

where J(t) is an abbreviation for the input. Since the first factor must be positive 
when a spike is being built up and the second factor must then also be positive so 
that the input contributes towards the spike, their product then is also positive 
during that phase. During the resetting phase, F and G are typically both 
negative, and so, we obtain again a positive contribution to the change of the 
weighted correlation. Thus, in this expression, a non-linear dependence of the 
change of state of the neuron on the present value of that state itself leads to a 
sharpening of the weighted input-output correlations through our spike timing 
dependent learning rule. 

These findings remain valid when we look at the precise formula in place of 1)28(1. 
That formula results from inserting l|20|l into \n\ : 

^E/ t\ GWt,.,+r)) (29) 

„ Jo J ~s 

poo -, psl 

T (mifc,A+ri)) 

J. Jo J^l J-si 

+G{'d{tk,x + Ti)) Yl ''(^fc.^ " ^'^p + (-r(si))dsi 

- tk,u + t) dr {~T{s))ds. 

V 

Here, the first sum extends over all spike times of neuron j with t — Ti< tj,f^ < t, 
whereas the second, inner, sum counts spikes of neuron k with + t — Ti < 
tk,x < tj^fj_ + T. So, the first G records the internal state of neuron i at the 
time tj^fj, + r, and the sum over the spike times of k records the weighted input- 
output correlation coming from k over the time period starting T/ earlier. Of 
course, we may assume as an approximation that only the weights of selected 
synapses change whereas the others remain stationary if we wish to analyze a 
situation where only those specific synapses receive a new input pattern whereas 
the weights of the others have settled in a stationary state in response to a sta- 
tionary input pattern. In such a situation, the sum over k then extends only 
over those neurons whose synapses undergo changes. In particular, we may con- 
centrate on the effect of the synapse from neuron j. - The role of the k terms 
has already been discussed above. 

It remains to treat the indirect effect coming from the variation of "d in 
response to the variation of w. satisfies a differential equation of the form 

^ = + ^'(i?)wK (30) 
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(thus, comparing this with we have u — F{§) + G{-d)wn — ^-d = 

^($(z?) + ^{^)wk)). Differentiating (|^ with respect to w, the variation 
w := 4^ satisfies 

aw 

UJ = <^'{'d)uJ + ^'{d)uJWK + ^{d)K. (31) 

Since we wish to understand the effect of learning during some time interval 
\t — /i, t], we assume that uj{t — h) = 0. We then obtain (for tq > t — h) 

= g/;_\(<E''(iJ(T))+*'(i/(T))«,(r)K(r))dr ^32) 

It-h 

Thus from H25|l . the indirect effect is 

d-d 

iF'{§)+G'i^)wK)—w, (33) 
with ^=uj from The more precise formula is, after letting h tend to 0, 



1 

Ti 



{F'{^{t,,^ + T)) (34) 

+ G"(l9(ij- ^ + r)) ^ W.,ra J2 '^'(^J'A' +T' *™,p)) 
rn p 

{'tj,!-^ + 'r)'^'^ik{t3.,fj. + t) dr {-T{s))ds 



UJ 



with UJ from H32|l and Wik from l|2()|l . Assuming 4'('!?) to be positive, ^ = uj can 
be discussed in the same manner as the direct effect, but the factor in front of 
it causes a somewhat different effect. Namely, while we may assume that G(z?) 
is positive when u is rising, and negative when it is falling, the derivative G' {d) 
will then naturally be positive only during the initial phase of the rise of u, but 
then turn negative to bring G{'d) back to at the firing value. Thus, here we see 
a positive effect in response to a weight change only during that initial phase, 
but a negative one already before the neuron fires. The reason for this is of 
course simply that the strengthening of the synapse will bring the neuron closer 
to the firing threshold in response to the corresponding input, and thereby push 
it into a state where it is less receptive to input, or, more precisely, where the 
input has a comparatively smaller effect. It will depend on the neuron model 
employed how close to the firing point that effect sets in. Likewise, F{d) may 
be negative over part or even all of the range of interest. 

In any case, we can then insert H20(l to evaluate the changes in the input- 
output correlations explicitly, although the complete formula will get somewhat 
complicated. Therefore, we have discussed here the qualitative effects instead. 
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5 Discussion 



We have introduced the learning rule 




-oo J/ Jt-Ti ^ 



where the are the arrival times of spikes from the presynaptic neuron j 
and the function u{t) describes the state of the postsynaptic neuron i. T(s) is 
the learning window, positive for negative, negative for positive values of s of 
sufficiently small absolute value, and satisfying the symmetry assumption 



We employ that state function u instead of is spike train because in general 
many presynaptic neurons contribute towards a spike of i and because an an- 
alytical treatment needs to incorporate the underlying rule how the input and 
the internal dynamics of a neuron generate its spike pattern whereas we can 
treat presynaptic spikes as external events. 

We could then study for general classes of neuron models how the relations 
between incoming spikes and the internal state of i determined the synaptic 
changes through our learning rule (|35|l . 

Moreover, we were able to derive an analytical expression for the converse effect, 
namely the changes in the input-output relationship caused by those synaptic 
weight changes. For that purpose, we needed a stationarity assumption that 
without learning, that relation would have been static. This seems the only 
reasonable way to isolate the effects of synaptic learning. 

As in most theoretical studies on learning through synaptic weight changes in 
neural networks, we have exclusively treated excitatory synapses although in- 
hibiting synapses could also be handled within our framework. 

Acknowledgement: I am grateful to the referee for his very detailed and 
insightful report that led to substantial improvements of this paper. 
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